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TECHNICAL REPORT 



The RICIS Concept 



The University of Houston -Clear Lake established the Research Institute for 
Computing and Information Systems (RICIS) In 19S6 to encourage the NASA 
Johnson Space Center (JSC) and local industry to actively support research 
in the computing and information sciences. As part of this endeavor. UHCL 
proposed a^artnership with JSC to jointly define and manage an integrated 
program of research in advanced data processing technology needed for JSC’s 
main missions, including administrative, engineering and science responsi- 
bilities. JSC agreed and entered into a continuing cooperative agreement 
with UHCL beginning in May 1986, to jointly plan and execute such research 
through RICIS. Additionally, under Cooperative Agreement NCC 9-16, 
computing and educational facilities are shared by the two institutions to 
conduct the research. 

The UHCL/ RICIS mission is to conduct, coordinate, and disseminate research 
and professional level education in computing and information systems to 
serve the needs of the government, industry, community and academia. 
RICIS combines resources of UHCL and its gateway affiliates to research and 
develop materials, prototypes and publications on topics of mutual interest 
to its sponsors and researchers. Within UHCL, the mission is being 
implemented through interdisciplinary involvement of faculty and students 
from each of the four schools: Business and Public Administration, Educa- 
tion, Human Sciences and Humanities, and Natural and Applied Sciences. 
RICIS also collaborates with industry in a companion program. This program 
is focused on serving the research and advanced development needs of 
industry. 

Moreover, UHCL established relationships with other universities and re- 
search organizations, having common research interests, to provide addi- 
tional sources of expertise to conduct needed research. For example, UHCL 
has entered into a special partnership with Texas A&M University to help 
oversee RICIS research anl education programs, while other research 
organizations are involved via the “gateway" concept 

A major role of RICIS then is to find the best match of sponsors, researchers 
and research objectives to advance knowledge in the computing and Informa- 
tion sciences. RICIS, working jointly with its sponsors, advises on research 
needs, recommends principals for conducting the research, provides tech- 
nical and administrative support to coordinate the research and integrates 
technical results into the goals of UHCL, NASA/JSC and industry. 
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This research was conducted under auspices of the Research Institute for Computing 
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coordinator. 
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Both research coordinators are from the Information Systems Directorate 
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1.0 Introduction : 


As part of the RICIS project # AR.06 activity, the reinforcement learning techniques 
developed at Ames Research Center are being applied to proximity and docking operations 
using the Shuttle and Solar Maximum Mission (SMM) satellite simulation. In utilizing these 
fuzzy learning techniques, we use the Approximate Reasoning based Intelligent Control 
(ARIC) architecture, and so we use these two terms interchangeably to imply the same. 
This activity is carried out in the Software Technology Laboratory utilizing the Orbital 
Operations Simulator (OOS) and programming / testing support from other contractor 
personnel. 

This report is the final deliverable D4 in our milestones and project activity. It provides the 
test results for the special testcase of approach/docking scenario for the shuttle and SMM 
satellite. Based on our experience and analysis with the attitude and translational 
controllers, we have modified the basic configuration of the reinforcement learning 
algorithm in ARIC. The shuttle translational controller and its implementation in ARIC is 
described in our deliverable D3. In order to simulate the final approach and docking 
operations, we have set-up this special testcase as described in section 2. The ARIC 
performance results for these operations are discussed in section 3 and conclusions are 
provided in section 4 alongwith die summary for the project 

2.0 Approach/Docking Tetstcase Description 

As a part of our third task, we had planned to evaluate the ARIC performance for the final 
approach and docking operations scenario. Typically, the shuttle approaches the satellite 
from 50 feet to less than 10 feet and continues to station keep at that distance for a while. 
The Shuttle robotic arm is maneuvered to grapple the satellite in a short time. Actually, the 
reach envelope of the robotic arm is around 35 feet, and thus, the arm is maneuvered into 
proper position during the approach and the satellite is captured during the tail end of the 
approach and the begining of the station-keeping phase. 

To simulate this scenario without the robotic arm, we have designed this testcase with three 
segments (figure 1). The first segment is a station keeping mode at a distance of 200 feet 
with a 15 feet range deadband (DB) and 5 ft/sec range rate DB. The shuttle attitude in the 
LVLH frame is (0, 90, 0) in the roll, pitch and yaw sequence, and remains the same in all 
three segments. The second segment is an approach from 200 feet to 10 feet distance with a 
range rate DB of 0.05 ft/sec and 2- 10 feet of range DB. Actually, the approach does not use 
range DB at all. It uses only the range rate DB and performs the approach task by 
maintaining the desired range rate. At the end of the approach segment at 10 feet distance, 
the desired range rate is 0.0 ft/sec with a 0.05 ft/sec of deadband. The third segment is 
station keeping at 10 feet distance with 1.25 ft range DB and 0.05 ft/sec range rate DB. If 
an automatic ARIC system can maintain this position, ( 10 feet range with zero elevation 
and zero azimuth angles and the orientation of 0, 90, 0 in roll, pitch and yaw respectively ) 
within their respective deadbands, then, it is performing proximity operations tasks very 
well. Please note that the ARIC controller has no constraints on fuel usage. 

In order to simulate this scenario, we had initially planned to begin the mission at 50 feet 
and approach all the way to 2 feet distance from the satellite. When we performed detailed 
calculations to see what are the effects of attitude deadband at 2 feet, we realized that this 
distance is very close to the satellite, especially with a medium DB used by the shuttle 
attitude controller. Considering the size of the SMM, a collision may not be avoided. Thus, 
we decided to stop at 10 feet distance from the satellite. We could begin the mission at 50 
feet distance, however, to stabilize the ARIC weights, particularly, the d's and f s in the 
Action Selection Network (ASN), a definite amount of learning is required by the ASN. 
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Figure 1 . Relative Trajectory for Approach and Docking 


Thus, we had to have some station keeping time before we can begin our approach. Since 
station keeping at 50 feet provides further problem with maintaining the elevation angle, it 
was best to start the testcase at 200 feet. The radar sensor is located approximately 40 feet 
from the center of mass of the shuttle, and thus, it would require approximately 14 degrees 
of elevation angle if proper range deadband is to be maintained. However, the cross-over 
for the Zero and Positive_Small membership functions of the elevation angle is at 8 
degrees. Thus, the elevation controller may violate the range deadband while correcting for 
high elevation angle and the range controller may violate the elevation deadband. There will 
be two opposing tasks for two axes of the translational controller both contributing to high 
fuel usage. To avoid this situation, we decided to start the mission at 200 feet where the 
elevation angle required is only 7 degrees, less than the cross-over for the elevation angle 
membership functions. Since the approach task does not use the range DB, there will be no 
conflict between the range and elevation control. 

Several changes and modifications have been made in the fuzzy learning algorithms within 
the ARIC framework. These changes were determined necessary based on our study so far 
to properly utilize the fuzzy learning techniques for space operations. The ARIC algorithm 
with these changes provides us a baseline for total 6 degree of freedom control. This 
includes attitude as well as translational modules : pitch, yaw, roll axes attitude control, and 
range, azimuth and elevation control along the line of sight 

a. We have removed the bias input from all modules. Thus all control networks ( Action 
Evaluation Network as well as ASN ) will have only two inputs and one output It is found 
that the bias term really impedes the learning in these networks. 

b. Updates of d's and f s weights in the ASN depend on the belief value of premise as well 
as consequent part of the rule. Thus, only those weights are updated for which rules fire. If 
the firing strength of the rule is zero, then, the weights are not updated. 

c. All failures in ARIC are crisp failures with protection so that the ASN performance does 
not degrade below an acceptable level. Our current implementation uses crisp failure, e.g. 0 
or -1. at 1.4 DB, but we protect the weight updating from too much punishment. 

d. If an action is modified by Stochastic Action Modifier, then, action is changed to no 
action rather than reverse action. 

e. There are no changes in computations for measure of confidence in ASN. Currently, the 
measure of confidence is a function of number of rules. This situation is debatable but for 
simplicity in our study, we have left it the same. 

f. Input parameters are normalized between 0.0 and 1.0. 

g. For the attitude controller, we will use overlapping Membership Functions as shown in 
figure 2. However, we will not utilize such overlap in rate errors for translational controller 
because we have not analyzed the performance without such overlap. At this time we do 
not know if a hysterisis exists in the translational control. 

3.0 Performance Results 

The control tasks during the proximity operations are : 1) maintain the desired range by 
controlling the thrust along the line of sight, 2) keep the elevation and azimuth angles close 
to zero with respect to the line of sight, and 3) maintain the desired LVLH attitude within 
the specified deadband. 
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Figure 2. Attitude Controller - Overlapping Rate Error MBF's. 
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It is expected that the attitude controller is performing with desired accuracy in maintaining 
the desired angle and rate. This assumption is necessary because the translational controller 
is slaved after the attitude controller in the shuttle proximity operations in manual mode. 

Our main results for the approach/docking special testcase are as follows. 

1. During the mission, the 6 DOF ARIC performs all required tasks and control for 
translation as well as rotation axes. In comparison it uses exetremely high amount of fuel. 
But, it should be noted that the ARIC has no fuel constraints. 

2. The orientation and relative position of the shuttle is accurately maintained during all 
segments of the mission. Figure 3 shows the pitch, yaw and roll angles in the LVLH frame 
during the mission. Figure 4 shows the x, y and z position in the LVLH frame. The x and z 
positions maintained at 10 and 45 feet respectively shows that the satellite range from the 
sensor is 10 feet and the elevation angle is zero because of sensor location on the shuttle. 
The y position is maintained within one foot distance indicates nearly zero azimuth angle. 

3. Roll and yaw rate coupling has high duty cycle and uses too much fuel. Figure 5 shows 
the body rates of the shuttle, and roll / yaw coupling seems to begin at 2000 seconds in the 
mission. It is appropriate to investigate the effects of overlap in rate membership functions 
for these axes. 

4. Range rate maintenance uses too much fuel, especially during the last segment of station 
keeping at 10 feet. This fuel rate could be decreased by changing the range rate DB from 
0.05 to 0. 1 ft/sec. Figure 6 shows the fuel usage as a function of time, and high fuel usage 
begins at 3500 seconds in the mission, the beginning of the station keeping phase at 10 feet 
distance. 

5. Azimuth maintenance is acceptable, but the deadband seems to be too tight. Further more 
the membership functions and ruels require close examination. 

4.0 Conclusions and Summary 

Based on the results obtained so far for the translational control as well as total 6 DOF 
control, it seems that the ARIC in its current implementation form is not suitable for 
proximity and docking operations. 

First of all it learns too slow because it does not encounter sufficient number of .failures. 
However, it is also true that the ARIC will never encounter sufficient number of failures for 
any space operations. Basically, all space operations are designed to be failure-free. Every 
failure is very costly in space operations. No single point failures are allowed by design. 
All activities in space are fail-safe, fail-operational, meaning, for all single point failures it 
has to be operational. Furthermore, the attitude going out of the deadband is never 
considered a failure. It is just one state that controller is able to control. Similarly, going out 
of rate deadband is not considered a failure. For use in ARIC acrhitecture, we can consider 
them as failures, but these are really soft failures. Thus, the failures or the rate of failures 
expected by the ARIC architecture are not going to occur operationally. It is for this reason, 
the current ARIC architecture is not suitable. 

Second, there is no mechanism in ARIC to sustain performance when two or more failures 
come in a sequence. There should be an automatic stop on learning and when a normal state 
is reached, an automatic trigger to learn. 
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Third, the ARIC architecture must allow soft failures and update weights accordingly. At 
this time we do not know what exactly is the impact of this change. However, for future 
testing with ARIC, we propose to implement failure value as shown in figure 7, and if 
possible, utilize a fuzzy rulebase that can consider fuel usage rate in computing this value. 

Fourth, the current membership functions definitions and defuzzification technique does 
not allow for the inertia coming from the zero rule and zero membership function. Thus, 
both, the membership functions definitions and the defuzzification method are inadequate. 

Fifth, there is no method in ARIC to move the end points of the membership functions. A 
capability to move these end points is required. 

Some of these concerns are taken care of in the new GARIC architecture. Hopefully, it will 
provide some insight into how to apply learning methods to the space operations. 



Appendix A. 

Plots of Selected Parameters for Shuttle Translational Control 


Al. First try with loose range and range rate deadbands. 
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SIMULATION APPLICATION: ARIC Trantlational Controller Simulation 
RUN IDENTIFICATION: 200 SK - Approach 10 - SK 
MODEL: ORBITER 
DATE: Wea Dec 23 1992 09:33:56 AM 
NUMBER OF DATA POINTS: 721 
DATA SAMPLING FREQUENCY: 0.100 Hz 
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SIMULATION APPLICATION: ARIC Translational Controller Simulation 
RUN IDENTIFICATION: 200 SK - Approach 10 - SK 
MODEL: ORBITER 
DATE: Wed Dec 23 1992 09:33:56 AM 

NUMBER OF DATA POINTS: 721 
DATA SAMPLING FREQUENCY: 0.100 Hz 
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Time (sec) 

SIMULATION APPLICATION: ARIC Translational Controller Simulation 
RUN IDENTIFICATION : 200 SK - Approach 10 - SK 
MODEL: ORBITER 
DATE: Wed Dec 23 1992 09:55:44 AM 

NUMBER OF DATA POINTS: 361 
DATA SAMPLING FREQUENCY: 0.050 Hz 
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learn cycle: Failure at 744.560000. 
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learn azim: Failure at 6992.640000. 
learn azim: Failure at 7021.440000. 
learn_azim: Failure at 7023.360000. 
learn azim: Failure at 7054.080000. 
learn_azim: Failure at 7056.000000. 
learn azim: Failure at 7084.800000. 
learn_azim: Failure at 7086.720000. 
learn azim: Failure at 7117.440000. 
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learn azim: Failure at 7152.000000. 
learn azim: Failure at 7153.920000. 
learn azim: Failure at 7182.720000. 
learn azim: Failure at 7184.640000. 

“ SIMULATION TERMINATED BY 'termin' 

SIMULATION START TIME: 0 

SIMULATION STOP TIME: 7.2e+03 

SIMULATION ELAPSED TIME: 7.2e+03 

ACTUAL ELAPSED TIME: 1.32e+03 

ACTUAL CPU TIME USED: 1.22e+03 

SIMULATION / ACTUAL TIME: 5.45 
SIMULATION / CPU TIME: 5.9 

CPU TIME FOR INITIALIZATION: 2.16 



A2. Second try with tight range and range rate deadbands. 
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SIMULATION APPLICATION: ARIC TnmtiMiond Controller Sumdation 
RUN IDENTIFICATION: 200 SK - Approach 10 - SK 
MODEL: ORBITER 
DATE: Tbe Dec 29 1992 10:05.07 AM 
NUMBER OF DATA POINTS: 721 
DATA SAMPLING FREQUENCY: 0.100 Hz 
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SIMULATION APPLICATION: ARIC Tranilalional Controller Simulation 
RUN IDENTIFICATION: 200 SK - Approach 10 - SK 
MODEL: ORBITER 
DATE: The Dec 29 1992 10:4758 AM 
NUMBER OF DATA POINTS: 721 
DATA SAMPLING FREQUENCY: 0.100 Hz 
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SIMULATION APPLICATION : ARIC Tnn.Utiontl Controller Simulation 
RUN IDENTIFICATION: 200 SR - Approech 10 -SR 
MODEL: ORBITER 
DATE: Tbe Dec 29 1992 10:47.58 AM 
NUMBER OF DATA POINTS: 721 
DATA SAMPLING FREQUENCY: 0.100 Hz 







Range ARIC Learning Parameters - Inputs 
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SIMULATION APPLICATION: ARIC Tnuwlational Controller Simulation 
RUN IDENTIFICATION: 200 SK - Approach 10 -SK 
MODEL: ORBITER 
DATE: Hie Dec 29 1992 10:06:20 AM 
NUMBER OF DATA POINTS: 721 
DATA SAMPLING FREQUENCY: 0.100 Hz 
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SIMULATION APPLICATION: ARIC Translational Controller Simulation 
RUN IDENTIFICATION: 200 SK - Approach 10 - SK 
MODEL: ORBITER 
DATE: T\ie Dec 29 1992 10:06:20 AM 

NUMBER OF DATA POINTS: 721 
DATA SAMPLING FREQUENCY: 0.100 Hz 
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SIMULATION APPLICATION: ARIC Translational Controller Simulation 
RUN IDENTIFICATION: 200 SK - Approach 10 - SK 
MODEL: ORBITER 
DATE: Die Dec 29 1992 10:06:20 AM 
NUMBER OF DATA POINTS: 721 
DATA SAMPLING FREQUENCY: 0.100 Hz 




Range ARIC Learning Parameters - General Parameters 


mui 'ifliTri iiiiii™ 



0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 5500 6000 6500 7000 7500 = 


Time (sec) 


Unusualness 


0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 5500 6000 6500 


Tune (sec) 


Failure 0 



—————I 


o 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 5500 6000 6500 7000 ^ 


Time (tec) 

SIMULATION APPLICATION : ARIC Trantlational Controller Simulation 

RUN IDENTIFICATION: 200 SK - Approach 10 - SK 
MODEL: ORBITER 
DATE: TUe Dec 29 1992 10:06:20 AM 
NUMBER OF DATA POINTS: 721 
DATA SAMPLING FREQUENCY: 0.100 Hz 
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MODEL: ORBITER 

DATE: Tie Dec 29 1992 10:07:39 AM 


NUMBER OF DATA POINTS: 361 
DATA SAMPLING FREQUENCY: 0.050 Hz 


B 





37 




